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Program description The Bilingual Cooperative Integrated Reading and Composition 

(BCIRC) program, an adaptation of the Cooperative Integrated 
Reading and Composition (CIRC) program, was designed to 
help Spanish-speaking students succeed in reading Spanish 
and then making a successful transition to English reading. In 


the adaptation, students complete tasks that focus on reading, 
writing, and language activities in Spanish and English, while 
working in small cooperative learning groups. The intervention 
focuses on students in grades 2-5. 


Research One study of BCIRC met the What Works Clearinghouse (WWC) students in second and third grades from seven schools in 
evidence standards with reservations. The study included 222 El Paso, Texas. 1 2 


Effectiveness 


BCIRC was found to have potentially positive effects on reading achievement and English language development. 


Rating of effectiveness 
Improvement index 2 


Reading achievement 

Mathematics achievement English language development 

Potentially positive 

na 

Potentially positive 

Average: +23 percentile points 

na 

Average: +11 percentile points 


na = not applicable 


1. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

2. A total of 85 students (52 in the treatment group, 33 in the comparison group) were posttested. These numbers show the average and range of improve- 
ment indices for all findings across the study. 
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Additional program 
information 


Research 
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Developer and contact 

BCIRC was adapted from CIRC by the study authors. CIRC was 
developed by Stevens, Madden, Slavin, and Famish (1987) 3 at the 
Center for Social Organization of Schools, Johns Hopkins Univer- 
sity. BCIRC is related to Alas Para Leer, an adaptation of Reading 
Wings for English language learners. In fact, BCIRC is now offered 
as Reading Wings on the Success for All website. All related pro- 
grams are distributed by Success for All Foundation, Inc. Address: 
200 W. Towsontown Boulevard, Baltimore, MD 21204-5200. Email: 
Dr. Madden at nmadden@successforall.org. Web: http://www. 
successforall.org . Telephone: (800) 548-4998 ext. 2372. 

Scope of use 

BCIRC is used as students make the transition from their pri- 
mary language to English language reading instruction in grades 
2-5. BCIRC was developed for use with students whose primary 
home language is Spanish. 


Teaching 

Teachers combined CIRC strategies with other transitional 
and English as a Second Language strategies to facilitate the 
development of language and reading skills in English. Several 
features were borrowed from CIRC to develop the BCIRC pro- 
gram. Fifteen activities occur before, during, and after reading, 
including using BCIRC materials to develop vocabulary, making 
predictions of a story’s content based on its title, and reading 
with a partner followed by silent reading. Teachers in the BCIRC 
program received extensive staff development on how to use a 
constructivist framework to facilitate student cooperative learn- 
ing discussions and discourse. 

Cost 

There is no information available on the cost of the intervention. 


One study reviewed by the WWC investigated the effects of 
BCIRC on the reading achievement and English language 
development of English language learners. The study 
(Calderon, Hertz-Lazarowitz, & Slavin, 1998) was a quasi- 
experimental design that met WWC evidence standards with 
reservations. 4 All students in the experimental schools (n = 3) 
and comparison schools (n = 4) were enrolled in bilingual 
programs and transitioning into English language instruction. 


Students in the comparison group participated in round-robin 
oral reading exercises and used workbooks for practice activi- 
ties. A total of 222 Spanish-speaking English language learners 
in two cohorts participated in the project. 5 However, only third 
graders were tested in English, so they are the only students 
included in this intervention report. At the time of posttesting, 
there were 85 third-grade students (n = 52 for BCIRC and 
n = 33 for control). 


3. Stevens, R. J., Madden, N. A., Slavin, R. E., & Famish, A. M. (1987). Cooperative Integrated Reading and Composition: Two field experiments. Reading 
Research Quarterly, 22, 433-454. 

4. The study authors are also the program developers. 

5. Two cohorts of students were involved in the study. Outcomes for only one cohort are discussed in this report, however, because students in the second 
cohort were not assessed on English language measures due to district policy and practice. 
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Effectiveness 


The WWC found BCIRC to 
have potentially positive 
effects on reading 
achievement and English 
language development 
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Findings 

The WWC review of interventions for English language learn- 
ers addresses student outcomes in three domains: reading 
achievement, mathematics achievement, and English language 
development. 

Reading achievement. Calderon, Hertz-Lazarowitz, and Slavin 
(1998) found statistically significant differences between the 
English language learners who participated in the BCIRC program 
and students in the comparison group. 6 The WWC analysis, how- 
ever, could not confirm the statistical significance of this finding 
because it was necessary to correct for clustering. The size of the 
effect was large enough to be considered substantively important, 
however, so the intervention in this study had potentially positive 
effects on reading achievement, according to WWC standards. 

English language development. Calderon, Hertz-Lazarowitz, 
and Slavin (1998) did not find statistically significant differences 


in English language development between the English language 
learners who participated in the BCIRC program and students 
in the comparison group. 7 The effect size, however, was large 
enough to be considered substantively important. So the inter- 
vention in this study had potentially positive effects on English 
language development, according to WWC standards. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, 8 the size of the 
difference between participants in the intervention and compari- 
son conditions, and the consistency in findings across studies 
(see the WWC Intervention Rating Scheme) . 


Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see Technical Details 
of WWC-Conducted Computations) . The improvement index rep- 
resents the difference between the percentile rank of the average 
student in the intervention condition versus the percentile rank 
of the average student in the comparison condition. Unlike the 
rating of effectiveness, the improvement index is entirely based 
on the size of the effect, regardless of the statistical significance 
of the effect, the study design, or the analyses. The improvement 
index can take on values between -50 and +50, with positive 
numbers denoting results favorable to the intervention group. 


The average improvement index is +23 percentile points 
for reading achievement and +11 percentile points for English 
language development for the one study reviewed. 

Summary 

The WWC reviewed one study on BCIRC, which met WWC 
evidence standards with reservations. Based on this study, the 
WWC found potentially positive effects on both reading achieve- 
ment and English language development. Specifically, the 
statistically significant findings for the percentage of students 
meeting the bilingual education exit criterion is an important 
and promising finding. The evidence presented in this report is 
limited and may change as new research emerges. 


6. At the end of third grade, 32% of students in the BCIRC group scored above the 40th percentile on the Norm-Referenced Assessment Program for Texas 
(NAPT) reading test, qualifying them to exit their bilingual education program. By contrast, only 10% of students in the comparison group met this criterion. 

7. At the end of third grade, 39% of students in the BCIRC group scored above the 40th percentile on the Norm-Referenced Assessment Program for Texas 
(NAPT) language test, qualifying them to exit their bilingual education program. By contrast, 21% of students in the comparison group met this criterion. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted 
Computations for the formulas the WWC used to calculate the statistical significance. In the case of BCIRC, a correction for clustering was needed. 
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References Met WWC evidence standards with reservations 

Calderon, M., Hertz-Lazarowitz, R., &Slavin, R. (1998). Effects of 
Bilingual Cooperative Integrated Reading and Composition on 
students making the transition from Spanish to English read- 
ing. Elementary School Journal, 99(2), 153-165. 


Additional source: 

Calderon, M., Hertz-Lazarowitz, R., Ivory, G., & Slavin, R. E. 
(1997). Effects of Bilingual Cooperative Integrated Reading 
and Composition on students transitioning from Spanish 
to English reading (Report No. 10). Baltimore, MD: Center 
for Research on the Education of Students Placed at Risk. 
(ERIC Document Reproduction Service No. ED405428). 


For more information about specific studies and WWC calculations, please see the WWC Bilingual Cooperative 
Integrated Reading and Composition Technical Appendices . 
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Appendix 


Appendix A1 

Study characteristics: Calderon, Hertz-Lazarowitz, & Slavin, 1998 (quasi-experimental design) 

Characteristic 

Description 

Study citation 

Calderon, M., Hertz-Lazarowitz, R., & Slavin, R. (1998). Effects of Bilingual Cooperative Integrated Reading and Composition on students making the transition from Spanish to 
English reading. Elementary School Journal, 99, 153-165. 

Participants 

The study included 222 Spanish-speaking English language learners in the second (n = 120) and third (n = 102) grades. The students’ primary home language was Spanish. 
A total of 85 third-grade students (52 in the treatment group, 33 in the comparison group) were posttested in English in reading and language. 1 Three intervention and four 
comparison schools participated in the two-year study. 2 

Setting 

English language learners who participated in the study attended seven elementary schools in the El Paso, Texas school district. Overall, 79% of students in the district were 
Hispanic and 27% had limited English proficiency. The schools selected for inclusion in the study had the highest rates of poverty and the lowest levels of student achievement 
among the schools in the district with Spanish-dominant English language learners. 

Intervention 

BCIRC students were assigned to cooperative learning teams consisting of four heterogeneously grouped students (that is, groups contained a mix of high, medium, and 
low achieving students). BCIRC attempts to promote student discussion and dialogue during cooperative learning activities designed to help students develop critical 
thinking and reading comprehension skills as well as the overall ability to use academic English. Activities include partner reading, recognition of key components of a story, 
vocabulary development, creative writing, and tasks designed to promote reading comprehension. Teachers model reading strategies — such as making and confirming a 
prediction — before, during, and after reading. Cooperative groups then apply the demonstrated strategy while attempting to comprehend stories selected from their classroom 
text. Students were taught for two hours each day. One half-hour of the two-hour instruction included English as a Second Language (ESL) instruction for both intervention 
and comparison groups. 

Stories for both intervention and comparison classrooms were selected from the Macmillan Campanitas de Ora Spanish basal reading series. By the middle of the second 
grade, students alternated every two weeks between the Spanish basal and the Macmillan Transitional Reading Program basal series in English. 

Comparison 

The comparison group included four schools matched to the three intervention schools on demographic characteristics and academic ranking within the district. Further, 
individual classes within the intervention schools were matched with classes in the comparison schools on mean pretest scores. Comparison group students used the Macmillan 
Campanitas de Oro Spanish basal reading series and began to alternate between the Spanish basal and the Macmillan Transitional Reading Program basal series in English each 
day. Students received the same amount of instruction (two hours, including one half-hour of ESL instruction) but used the teachers’ editions of the McMillan reading series for 
guidance rather than the BCIRC approach. Overall, teachers in the comparison condition were trained in and used round-robin oral reading and workbook practice activities. 

Primary outcomes 
and measurement 

The effects of the intervention on English language learner outcomes were assessed using the Norm-Referenced Assessment Program for Texas (NAPT). Although the Texas 
Assessment of Academic Skills (TAAS) was used in the study to assess reading outcomes, results are not reported here because the measure was administered in Spanish. 
(See Appendices A2.1 and A2.2 for a more detailed description of the outcome measures.) 

Teacher training 

Teachers implementing the intervention received extensive staff development, but more specific information about the training was not provided. Teachers in comparison 
schools received training related to cooperative learning. 


1. There were 102 third-grade students who were pretested (64 in the treatment group, 38 in the comparison group). This sample loss does not exceed WWC limits for study attrition. 

2. Two cohorts of students participated in the study. However, data from only one of the third-grade English language learner cohorts are reported, because students in the other cohort were not 
assessed in English. 
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Appendix A2.1 Outcome measures in the reading achievement domain 


Outcome measure 

Description 

Norm-Referenced 
Assessment 
Program for Texas 
(NAPT) — Reading Scale 

English language learner outcomes in reading were assessed using the NAPT at the end of the third grade (as cited in Calderon, Hertz-Lazarowitz, & Slavin, 1998). The NAPT 
yields scores in reading, writing, mathematics, science, and social studies in English. 

Percentage of students 
who met exit criterion 
from bilingual education 

English language learner outcomes in reading were based on the exit criterion from bilingual education. Students who scored above the 40th percentile on the NAPT reading 
test met the exit criterion for reading achievement. 


Appendix A2.2 Outcome measures in the English language development domain 


Outcome measure 

Description 

Norm-Referenced 
Assessment 
Program for Texas 
(NAPT) — Language Scale 

English language learner outcomes in English language development were assessed using the NAPT at the end of the third grade (as cited in Calderon, Hertz-Lazarowitz, & 
Slavin, 1998). The NAPT yields scores in reading, writing, mathematics, science, and social studies in English. 

Percentage of students 
who met exit criterion 
from bilingual education 

English language learner outcomes in English language development were based on the exit criterion from bilingual education. Students who scored above the 40th percentile 
on the NAPT language test met the exit criterion for English language development. 
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Appendix A3.1 


Summary of study findings included in the rating for the reading achievement domain 1 





Author’s findings from the study 







Mean outcome 2 
(standard deviation 3 ) 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(students) 

BCIRC Comparison 

group group 

Mean difference 4 
(BCIRC - 
comparison) 

Statistical 
significance 6 
Effect size 5 (at a = 0.05) 

Improvement 

index 7 



Calderon, Hertz-Lazarowitz, & Slavin, 1998 (quasi-experimental design) 8 




Norm-Referenced Assessment Grade 3 

Program for Texas — Reading 

85 33.16 23.83 9.33 

(15.44) (14.98) 

0.61 

ns 

+23 

Domain average 9 for reading achievement 


0.61 

ns 

+23 


ns = not statistically significant 

1. This appendix reports findings considered for the effectiveness rating and the average improvement index. Subgroup findings from the same study are not included in these ratings, but are reported in Appendix A4.1. 

2. Adjusted means are reported. Kindergarten English and Spanish scores on the Bilingual Syntax Measure served as a pretest covariate. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the 
clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Calderon, Hertz-Lazarowitz, and Slavin 
(1998), a correction for clustering was needed, so the significance levels differ from those reported in the original study. 

9. This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated 
from the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the English language development domain 1 





Author’s findings from the study 







Mean outcome 2 
(standard deviation 3 ) 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(students) 

BCIRC Comparison 

group group 

Mean difference 4 
(BCIRC - 
comparison) 

Statistical 
significance 6 
Effect size 5 (at a = 0.05) 

Improvement 

index 7 



Calderon, Hertz-Lazarowitz, & Slavin, 1998 (quasi-experimental design) 8 




Norm-Referenced Assessment Grade 3 

Program for Texas — Language 

85 34.90 30.36 4.54 

(15.69) (15.91) 

0.29 

ns 

+11 

Domain average 9 for English language development 


0.29 

ns 

+11 


ns = not statistically significant 

1. This appendix reports findings considered for the effectiveness rating and the average improvement index. Subgroup findings from the same study are not included in these ratings, but are reported in Appendix A4.2. 

2. Scores are normal curve equivalents, and adjusted means were provided by the study authors. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . Though it is unclear why students in cohort 1 had either one year or two years of BCIRC exposure, the issue of two third-grade 
subsamples does not influence the effect size calculations or ratings presented in the report. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the 
clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Calderon, Hertz-Lazarowitz, and Slavin 
(1998), no corrections for clustering or multiple comparisons were needed. 

9. This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated 
from the average effect size. Though it is unclear why students in cohort 1 had either one or two years of BCIRC exposure, the issue of two third-grade subsamples does not influence the effect size calculations or ratings presented in 
the report. 
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Appendix A4.1 


Summary of subgroup findings for the reading achievement domain * 1 





Author’s findings from the study 







Mean outcome 2 
(standard deviation 3 ) 


WWC calculations 


Outcome measure 

Study 

sample 4 

Sample size 
(students) 

BCIRC Comparison 

group group 

Mean difference 5 
(BCIRC - 
comparison) 

Statistical 
significance 7 
Effect size 6 (at a = 0.05) 

Improvement 

index 8 


Calderon, Hertz-Lazarowitz, & Slavin, 1998 (quasi-experimental design) 9 


Norm-Referenced Assessment 
Program for Texas — Reading 

Grade 3— 
two years 

59 

36.83 

(nr) 

23.83 

(14.98) 

13.00 

0.87 

Statistically 

significant 

+31 

Norm-Referenced Assessment 
Program for Texas — Reading 

Grade 3— 
one year 

59 

28.83 

(nr) 

23.83 

(14.98) 

5.00 

0.33 

ns 

+13 

Percentage of students who 

Grade 3 

118 

0.32 

0.10 

0.22 

0.87 

Statistically 

+31 


met exit criterion from bilingual significant 

education — Reading 


ns = not statistically significant 

nr = not reported 

1 . This appendix presents subgroup findings for measures that fall in the reading achievement domain. Findings for the full sample were used for rating purposes and are presented in Appendix A3.1 . 

2. Scores are normal curve equivalents, and adjusted means are provided. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The study 
authors did not provide standard deviations for subgroups. 

4. “One year” represents students who were in the program for one year, and “two years” represents students who were in the program for two years. The study is unclear about why some third-grade students in cohort 1 had one year of 
BCIRC exposure while other third-grade students in the same cohort had two years of exposure. 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. The appendix table reports the effect sizes, but the WWC could not confirm the effect sizes because the study did not report standard deviations for the subgroups. The effect sizes reported by the study authors were computed as the 
difference in adjusted scores on the posttest divided by unadjusted control group standard deviations, which differs from the method that the WWC uses to compute effect sizes. For an explanation of the effect size calculation, see 

Technical Details of WWC-Conducted Computations . 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. The WWC could not confirm that the effects of the intervention were statistically signifi- 
cant because the study did not include standard deviations for the subgroups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

9. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Calderon, Hertz-Lazarowitz, and Slavin (1998), a correction for clustering was needed, which did not change the statistical significance of the findings reported by the study author for students who 
had been involved with the intervention for two years. However, the findings for students who had been involved with the intervention for one year became nonsignificant after correcting for clustering. 
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Appendix A4.2 Summary of subgroup findings for the English language development domain * 1 





Author’s findings from the study 







Mean outcome 2 
(standard deviation 3 ) 


WWC calculations 


Outcome measure 

Study 

sample 4 

Sample size 
(students) 

BCIRC Comparison 

group group 

Mean difference 5 
(BCIRC - 
comparison) 

Statistical 
significance 7 
Effect size 6 (at a = 0.05) 

Improvement 

index 8 


Calderon, Hertz-Lazarowitz, & Slavin, 1998 (quasi-experimental design) 9 


Norm-Referenced Assessment 
Program for Texas — Language 

Grade 3— 
two years 

59 

36.27 

(nr) 

30.21 

(nr) 

6.06 

0.38 

ns 

+15 

Norm-Referenced Assessment 
Program for Texas — Language 

Grade 3— 
one year 

59 

33.73 

(nr) 

30.21 

(nr) 

3.52 

0.22 

ns 

+9 

Percentage of students who 

Grade 3 

118 

0.39 

0.21 

0.18 

0.53 

ns 

+20 


met exit criterion from bilingual 
education — Language 


ns = not statistically significant 

nr = not reported 

1 . This appendix presents subgroup findings for measures that fall in the English language development domain. Findings for the full sample were used for rating purposes and are presented in Appendix A3. 2. 

2. Scores are normal curve equivalents, and adjusted means are provided. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The study 
authors did not provide standard deviations for subgroups. 

4. “One year” represents students who were in the program for one year, and “two years” represents students who were in the program for two years. 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. The effect sizes were provided by the study authors but could not be confirmed by the WWC because the study did not report standard deviations for the subgroups. The effect sizes reported by the study authors were computed as 
the difference in adjusted scores on the posttest divided by unadjusted control group standard deviations, which differs from the method that the WWC uses to compute effect sizes. Effect sizes for binary measures (for example, the 
percentage of students who met exit criterion on the NAPT) were calculated using a log odds ratio with Cox adjustment. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

9. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of Calderon, Hertz-Lazarowitz, and Slavin (1998), a correction for clustering was needed, so the significance levels for students who were involved in the intervention for two years differ from those 
reported by the study authors. The authors did not report statistically significant findings for students who were involved in the intervention for one year. 
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Appendix A5.1 Bilingual Cooperative Integrated Reading and Composition rating for the reading achievement domain 


The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of reading achievement, the WWC rated BCIRC as having potentially positive effects. It did not meet the criteria for positive effects because 
it had only one study. The remaining ratings (mixed effects, no discernible effects, potentially negative effects, and negative effects) were not considered because 
BCIRC was assigned the highest applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. BCIRC met this criterion because it had substantively important positive findings. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. BCIRC met this criterion because the one study reviewed did not show a statistically significant or substantively important negative effect or 
indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. BCIRC did not meet this criterion because only one study was reviewed. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. BCIRC met this criterion because the one study reviewed did not show statistically significant or substantively important negative effects. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.2 Bilingual Cooperative Integrated Reading and Composition rating for the English language development domain 


The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of English language development, the WWC rated BCIRC as having potentially positive effects. It did not meet the criteria for positive 
effects because it had only one study. The remaining ratings (mixed effects, no discernible effects, potentially negative effects, and negative effects) were not consid- 
ered because BCIRC was assigned the highest applicable rating. 


Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. BCIRC met this criterion because it had substantively important positive findings. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. BCIRC met this criterion because the one study reviewed did not show a statistically significant or substantively important negative effect or 
indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. BCIRC did not meet this criterion because only one study was reviewed. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. BCIRC met this criterion because the one study reviewed did not show statistically significant or substantively important negative effects. 


1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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